1. Field of the Invention
The present invention relates to image processing apparatus and method to display images on a display screen, edit and process the images displayed, and retrieve the images edited and processed, and a computer-readable recording medium with a program thereon for making the computer execute the method. In particular, the invention relates to a technique for joining plural images and correcting distortion of the images.
2. Discussion of the Background
There has conventionally been known a technique for joining plural images together to form an image picture. Such a technique would be used in a case where two or more images are necessary to photograph a panoramic view, such as the entire image of a tall building or an event hall, an aerial photograph or a case where the distance from an object to be photographed is too near for a normal digital still camera to put the entire image in a picture, such as an interior photo. In these cases, two or more pictures need to be joined with each other to form the panoramic view. A method of joining two images has been known in which two images are joined with each other by moving and putting either of the images in position by hand while viewing the joining part on a screen.
It is known that an image photographed by a digital still camera or video camera generally contains geometrical distortion produced by distortion aberration of the lens system forming the image in a position deviated from the original position. To correct distortion of such an image, a method is known such as one disclosed in Japanese patent application laid-open publication No. 9-294225, entitled xe2x80x9cMethod for Deciding Parameter for Image Distortion Correction and Image Pickup Device.xe2x80x9d This publication discloses a technique in which plural images including a common pattern picked up from one point are used to detect plural sets of corresponding observing points from these images and measure angles of the observation points with respect to an optical axis of the image pickup system, thus estimating a parameter for distortion correction based on the angle information obtained.
The above method of joining plural images together to form an image picture may be useful for skilled persons who are specialized in this work, but it is very hard for beginners to do these jobs efficiently. If two or more images are joined together to form an image picture, adjacent images need to be adjusted one after the other, respectively, and this makes it difficult even for skilled persons to do this work efficiently. It is also necessary to perform additional image correction so as to make the joining part inconspicuous. When the images are joined by hand, the image correction must also be performed by hand, and this results in a reduction in operation efficiency.
Image distortion correction requires detection of plural sets of corresponding observing points from images, the distortion of which should be corrected. It may be possible for skilled persons to detect observing points and input data related to the observing points, but it is very hard for beginners to do these jobs efficiently. When image distortion correction needs to be performed for a large number of images, detection of observing points and data input of the observing points are required for each image. This makes it difficult even for skilled persons to do these jobs efficiently.
In order to solve the above conventional problems, an object of the present invention is to provide an image processing apparatus and method capable of joining plural images in an easy operation even for beginners, and a computer-readable recording medium with a program recorded thereon for making the computer execute the method.
Another object of the present invention is to provide image processing apparatus and method capable of correcting distortion of images in an easy operation even for beginners, and a computer-readable recording medium with a program recorded thereon for making the computer execute the method.
In one aspect of the present invention, an image processing apparatus is provided with a display device to display images on a display screen so that the images displayed on the display device can be edited and processed for smooth joining of the images. The image processing apparatus includes an image specifying device to specify two or more images to be joined together on the display device. An image arranging device arranges the images specified by the image specifying device in such an order as to join the images together, and a display control device controls the display device to display the images arranged by the image arranging device. A joining point specifying device specifies a joining point for each image so that vertically or laterally adjacent images displayed by the display control device can be joined together by referring to the joining points. A joining device joins adjacent images together by referring to the joining points specified by the joining point specifying device. According to the present invention, the specified images are arranged in such an order as to be joined together, and the arranged images are so displayed that a joining point can be specified for each image while viewing both images displayed adjacent to each other. This makes it possible to easily join the images together by merely specifying one joining point for each image, and hence to execute editing and processing of the images such as image joining easily and efficiently.
In another aspect of the present invention, the image processing apparatus further includes a connection line drawing device to draw a connection line between joining points specified by the joining point specifying device, and the display control device controls the display device to display both the images arranged by the image arranging device and the connection line drawn by the connection line drawing device. According to the present invention, a connection line is drawn between the joining points specified, so that the operator can both specify the joining points as if he or she drew a line by hand, and recognize instantaneously whether the joining points are specified or not.
In still another aspect of the present invention, the image processing apparatus further includes a joining-point specification releasing device to release specification of the previously specified set of joining points when a given set of joining points is specified for a pair of adjacent images. According to the invention, when a set of joining points is specified for a pair of adjacent images for which another set of joining points has already been specified, specification of the previous set of joining points is released, so that when specifying a new set of joining points, the operator can change the specification of joining points easily and efficiently in the same operation as when specifying a set of joining points for the first time, without the need to release the specification of the previous set of joining points.
In yet another aspect of the present invention, the image processing apparatus further includes a focal length setting device to set the focal length with which an image has been input, and an image correcting device to correct the image based on the focal length set by the focal length setting device. According to the invention, the focal length with which an image has been input is set to correct the image based on the focal length set for joining the image with another. This makes it possible to join images without distortion, even if the images have been input by input devices with different focal lengths.
In still another aspect of the present invention, a method for controlling an image processing apparatus for joining of images includes steps of displaying images on a display screen, specifying two or more images to be joined together, arranging the specified images in such an order as to be joined together, displaying the arranged images, specifying a joining point for each image so that adjacent images displayed can be joined by referring to the joining points, and joining adjacent images together by referring to the specified joining point. According to the invention, the specified images are arranged in such an order as to be joined together, and the arranged images are so displayed that a joining point can be specified for each image while viewing both images displayed adjacent to each other. This makes it possible to join the images by a simple operation of specifying one joining point for each image, and hence to execute editing and processing of the images, such as image joining, easily and efficiently.
In still another aspect of the present invention, the method for joining of images further includes steps of drawing a connection line between the specified joining points, and displaying not only the arranged images but also the drawn connection line. According to the invention, a connection line is drawn between the joining points specified, so that the operator can specify both the joining points as if he or she drew a line by hand, and recognize instantaneously whether the joining points are specified or not.
In yet another aspect of the present invention, the method for joining of images further includes a step of releasing specification of the previously specified set of joining points when a new set of joining points is specified. According to the invention, when a set of joining points is specified for a pair of adjacent images for which another set of joining points has already been specified. specification of the previous set of joining points is released, so that when specifying a new set of joining points, the operator can change the specification of joining points easily and efficiently in the same operation as when specifying a set of joining points for the first time, without the need to release the specification of the previous set of joining points.
In yet another aspect of the present invention, the method for joining of images further includes a step of setting the focal length with which an image has been input, and a step of correcting the image based on the focal length set for use in joining the image with another. According to the invention, the focal length with which an image has been input is set to correct the image based on the focal length set for joining the image with another. This makes it possible to join images without occurrence of distortion even if the images have been input by input devices with different focal lengths.
In yet another aspect of the present invention, a computer readable recording medium stores computer instructions for controlling an image processing apparatus for joining of images by performing the above-described steps. This makes it possible for a computer to read the computer instructions, and hence for the computer to carry out the method for controlling an image processing apparatus for the joining of images.
In still another aspect of the present invention, an image processing apparatus is provided with a display device for displaying images on a display screen so that the images displayed on the display device can be edited and processed for correction of image distortion. An image specifying device specifies at least two images which are photographed at slightly different photographing positions for correcting distortion of the images. An image arranging device arranges the images specified by the image specifying device, and a display control device controls the display device to display the images arranged by the image arranging device. An identical point specifying device specifies an identical point for each image displayed by the display control device, and a distortion correcting device corrects distortion of images by referring to the identical points specified by the identical point specifying device. According to the invention, image distortion can be corrected by a simple operation to merely specify one identical point for each image displayed, thus editing and processing the images easily and efficiently.
In yet another aspect of the present invention, the image processing apparatus for correction of image distortion further includes a connection line drawing device to draw a connection line between identical points specified by the identical point specifying device, and the display control device controls the display device to display both the images arranged by the image arranging device and the connection line drawn by the connection line drawing device. According to the invention, a connection line is drawn between the identical points specified, so that the operator can specify the identical points as if he or she drew a line by hand, and recognize instantaneously whether the identical points are specified or not.
In still another aspect of the present invention, the image processing apparatus for correction of image distortion further includes an identical point specification releasing device to release specification of the previously set of identical points when a new set of identical points is specified for images for which another set of identical points has already been specified. According to the invention, when a set of identical points is specified for images for which another set of identical points has already been specified specification of the previous set of identical points is released, so that when specifying a new set of identical points, the operator can change the specification of identical points easily and efficiently in the same operation as when specifying a set of identical points for the first time without the need to release the specification of the previous set of identical points.
In still another aspect of the present invention the image processing apparatus for correction of image distortion further includes a focal length setting device to set a focal length with which an image has been input, and the distortion correcting device corrects distortion of the image based on the focal length set by the focal length setting device. According to the invention, the focal length with which an image has been input is set to correct distortion of the image based on the focal length set. This makes it possible to perform proper distortion correction of images even if the images have been input by input devices with different focal lengths.
In yet another aspect of the present invention, a method for controlling an image processing apparatus for correction of image distortion includes steps of displaying images on a display screen, specifying at least two images photographed at slightly different photographing positions, for correcting distortion of the images, arranging the images specified in the image specifying step, controlling display of the images arranged in the image arranging step, specifying an identical point for each image displayed in the display control step, and correcting distortion of the images by referring to the identical points specified in the identical point specifying step. According to the invention, image distortion can be corrected by a simple operation to merely specify one identical point for each image displayed, thus editing and processing the images easily and efficiently.
In yet another aspect of the present invention, the method for correction of image distortion further includes a step of drawing a connection line between identical points specified in the identical point specifying step, and the display control step controls display of both the images arranged in the image arranging step and the connection line drawn in the connection line drawing step. According to the invention, a connection line is drawn between the identical points specified, so that the operator can both specify the identical points as if he or she drew a line by hand, and recognize instantaneously whether the identical points are specified or not.
In still another aspect of the present invention, the method for correction of image distortion further includes a step of releasing specification of the previous set of identical points when a set of identical points is specified for images for which another set of identical points has already been specified. According to the invention, when a set of identical points is specified for images for which another set of identical points has already been specified, specification of the previous set of identical points is released, so that when specifying a new set of identical points, the operator can change the specification of identical points easily and efficiently in the same operation as when specifying a set of identical points for the first time without the need to release the specification of the previous set of identical points.
In still another aspect of the present invention, the image processing method for image distortion correction further includes a step of setting a focal length with which an image has been input, and the distortion correcting step corrects distortion of the image based on the focal length set in the focal length setting step. According to the invention, the focal length with which an image has been input is set to correct distortion of the image based on the focal length set. This makes it possible to perform proper distortion correction of images even if the images have been input by input devices with different focal lengths.
In still another aspect of the present invention, a computer readable recording medium stores computer instructions for controlling an image processing apparatus for image distortion correction by performing the above-described steps. This makes it possible for a computer to read the instructions mechanically, and hence for the computer to carry out the method for controlling an image processing apparatus for the image distortion correction.